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Chapter 


Introduction 


What  is  the  OPS  TEST  Application? 

The  Institute  for  Information  Technology  Applications’  Operational  Test 

and  Evaluation  Support  Tool  (OPS  TEST)  is  an  automated  testing 

environment  that  includes  a  repository  of  training  and  evaluation  test 

items  and  tests.  Specific  functions  provided  are: 

•  Ability  to  add  new  and  update  existing  test  items,  which  are  held  as 
Microsoft  Word  documents  with  their  associated  reference 
information. 

•  Ability  to  build  new  and  maintain  existing  master  tests  and  to  create 
specific  instances  of  these  tests  for  crew  members  to  take. 

•  Ability  to  selectively  retrieve,  track,  and  record  test  results. 

•  Ability  to  selectively  analyze  the  performance  of  specific  test  items 
to  assess  their  quality  and  effectiveness. 

•  Ability  of  crew  members  to  take  formal  training  and  evaluation  tests 
on-line  with  automatic  grading  and  feedback  to  the  test-taker. 

•  Ability  of  crew  members  to  access  the  Master  Question  Files  and  to 
build  and  take  practice  tests  from  these  files  with  immediate 
feedback  on  each  question. 


Intended  Audience 

This  manual  is  intended  for  several  audiences  to  include: 

•  Test  Administrators:  Primarily  training  and 

standardization/evaluation  personnel  responsible  for 
building,  maintaining,  tracking,  and  analyzing  formal  tests. 
These  individuals  will  be  interested  in  the  entire  manual. 
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•  Testbank  Maintainers:  Those  tasked  to  maintain  the 

OPS  TEST  database.  These  individuals  will  be  interested 
in  the  entire  document. 

Test-takers  would  be  interested  in  Chapters  7,  8,  and  9  of  this  manual, 
but  are  better  served  by  the  OPS  TEST  Test-Taker’s  Manual  (Volume 
III  in  the  OPS  TEST  documentation),  which  is  specifically  designed  for 
their  needs. 

System  Requirements 

OPS  TEST  is  an  Access  97  application  intended  to  run  within  a 
Microsoft  Office  97  environment  to  include  an  installed  version  of 
Access  97.  OPS  TEST  requires  a  Windows  95/98/NT/2000  capable 
personal  computer,  preferably  a  Pentium  (or  faster)  with  at  least  16MB 
of  main  memory  (RAM)  and  3MB  of  available  disk  space.  A  monitor 
running  with  a  resolution  of  at  least  800  X  600  is  also  assumed.  Using 
a  lower  resolution  will  result  in  some  forms  not  being  fully  displayed  and 
accessible. 

Overview 

The  OPS  TEST  consists  of  two  primary  areas:  Administration  and 
Test-Taking.  OPS  TEST  Administration  is  oriented  toward  the 
standardization/evaluation  and  training  personnel  responsible  for  the 
development  of  training  and  evaluation  tests.  In  fact,  there  are  two 
different  versions  of  OPS  TEST  Administration,  one  for 
standardization/evaluation  and  one  for  training.  The  standardization/ 
evaluation  version  allows  access  to  all  questions  in  the  testbank, 
however,  the  training  version  disallows  access  to  the  questions  in  the 
“Eval”  category. 

The  OPS  TEST  Administration  functions  are  under  the 
START|PROGRAMS|OPS  TEST  ADMINISTRATION  menu  from 
Windows. 


Once  within  the  OPS  TEST  Administration  program,  the  main  menu 
shown  in  Figure  1.1  is  available  and  provides  access  to  1)  add  new  and 
update  existing  test  items,  2)  build  and  maintain  existing  master  tests 
and  specific  tests  made  from  the  master  tests,  3)  view  the  results  and 
status  of  pending  specific  tests,  and  4)  perform  analysis  of  how 
particular  test  items  have  performed  across  their  usages.  The  test 
building  paradigm  is  for  master  tests  to  be  build  once  and  then  copies 
made  of  this  master  test  to  be  taken  by  specific  crew  members  (specific 
tests). 
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The  OPS  TEST  Administration  software  also  provides  menu  bar 
functions  to  configure  parameters  for  the  forms  and  to  maintain 
common  information  like  personnel  data,  task  codes,  operations 
centers,  and  crew  positions.  These  functions  are  further  described  in 
Chapter  10. 


Figure  1.1:  OPS  TEST  Administration  Main  Menu 

The  crew  member  test-taking  program  is  accessed  via  the  Windows’ 

START  I  PROGRAMS  |  OPS  TEST  TEST-TAKTNG  menu.  The  main  test¬ 
taking  menu  is  shown  as  Figure  1.2.  The  first  button  provides  access  to  the 
formal  training  and  evaluation  specific  tests  created  for  crew  members.  These 
tests  are  uniquely  build  for  individual  crew  members  and  are  password  protected 
for  security.  When  taking  these  tests,  crew  members  are  presented  with  one  test 
item  at  a  time  and  are  allowed  to  move  back  and  forth  through  the  entire  test 
answering  or  not  answering  test  items  as  they  prefer.  Specific  buttons  provide 
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quick  access  to  un-answered  test  items.  When  the  test-taker  has  completed  the 
test,  they  press  a  button  to  have  it  graded.  If  they  have  any  un-answered  test 
items,  they  are  given  the  opportunity  to  go  back  and  answer  them  prior  to 
grading.  Once  the  test-taker  has  the  test  graded,  they  are  told  their  score  and 
given  the  opportunity  to  review  any  test  items  they  missed.  This  review  shows 
them  both  their  answer  and  the  correct  answer  together  with  the  associated 
reference  and  feedback  information  for  the  test  item. 


Figure  1.2:  Main  Test-Taking  Menu 


The  second  option  provides  access  to  training  tests  that  are  designed  to  be 
created  from  a  master  test  upon  demand.  These  tests  have  more  flexibility  in  that 
they  do  not  need  to  be  built  for  specific  individuals  ahead  of  time  and  may  be  set 
to  be  re-entered  multiple  times. 

The  third  test-taking  option  is  for  crew  members  to  access  the  Master  Question 
File  (MQF)  on-line.  In  this  function,  they  are  provided  a  query  form  to  use  in 
selecting  test  items  from  the  MQF.  They  then  answer  these  questions  one  at  a 
time  and  are  provided  immediate  feedback  as  to  whether  or  not  they  correctly 
answered  the  test  item.  If  they  answered  incorrectly,  they  are  provided  both 
associated  references  and  the  item’s  feedback  information.  They  are  also 
provided  a  final  summary  of  their  results  to  include  an  overall  score  for  that  set  of 

test  items. 
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Chapter 


Installation  and  Support 


How  do  I  get  OPS  TEST  up  and  running?  What 
do  I  do  if  I  need  heip? 


As  mentioned  in  Chapter  1 ,  there  are  two  parts  to  the  OPS  TEST.  OPS 
TEST  Test-Taking  provides  an  automated  mechanism  for  taking  formal 
training  and  evaluation  tests,  monthly  training  tests,  and  for  accessing 
the  Master  Question  File  for  answering  practice  test  questions.  The 
other  part  of  OPS  TEST  is  OPS  TEST  Administration,  which  is  only 
accessible  to  standardization/evaluation  and  training  personnel. 
Installation  procedures  for  both  parts  are  provided  below. 

These  instructions  assume  that  you  have  the  OPS  TEST  database 
installed  and  configured  on  your  network.  If  this  is  not  the  case,  you 
need  to  perform  the  initial  database  installation  and  configuration 
procedures  described  in  Chapter  10  of  this  document. 

OPS  TEST  Test-Taking  Installation 

Installation  of  OPS  TEST  Test-Taking  can  be  accomplished  using  the 
installation  compact  disk  (CD).  To  accomplish  installation  of  OPS 
TEST  Test-Taking  via  the  CD  use  the  instructions  below: 

1 .  Place  the  OPS  TEST  Test-Taking  installation  CD  into  your  CD-ROM 
drive. 

2.  Select  START|RUN  from  Windows  and  enter: 
“d:\TestTaking\Setup.exe”  (replace  the  “d:”  as  appropriate  if  your 
CD-ROM  uses  a  different  device  letter). 

3.  Select  all  the  default  options  presented  to  you. 

4.  When  you  initially  start  up  the  OPS  TEST  Test-Taking  program,  you 
will  be  prompted  for  the  location  of  the  OPS  TEST  Backend 
Database.  Your  OPS  TEST  Administrator  will  provide  you  with  this 
path/location.  You  may  also  be  prompted  for  a  password  for  the 
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OPS  TEST  Backend  Database.  This  also  can  be  provided  by  your 
OPS  TEST  Administrator. 


OPS  TEST  Administration  Installation 

Installation  of  OPS  TEST  Administration  is  only  provided  to  appropriate 
standardization/evaluation  and  training  personnel  and  thus  the 
installation  CDs  are  tightly  controlled  and  the  program  is  password- 
protected.  Furthermore,  there  are  separate  versions  of  OPS  TEST 
Administration  for  standardization/evaluation  and  training.  You’ll  need 
to  see  personnel  in  those  offices  for  access  to  the  installation  CDs  and 
the  initial  passwords.  Once  you  have  the  CD,  you  can  install  OPS 
TEST  Administration  via  the  instructions  below.  Also,  they  may  have  a 
copy  of  the  installation  CD  on  their  restricted  access  shared  drives. 

See  them  for  exact  locations. 

Once  you  install  OPS  TEST  Administration  and  open  it  using  the  initial 
password,  you  can  change  the  password  on  your  installation  via 
functions  within  the  Administration  software. 

1 .  Install  the  OPS  TEST  Administration  installation  CD  into  your  CD- 
ROM  drive. 

2.  Select  START|RUN  from  Windows  and  enter: 
“d:\EvalAdmin\Setup.exe”  or  “d:\TrngAdmin\Setup.exe”  depending 
on  whether  the  evaluation  or  training  version  of  the  Administration 
component  is  to  be  installed.  Replace  the  “d:”  with  the  drive  letter  of 
your  CD-ROM  as  necessary. 

3.  Select  all  the  default  options  presented  to  you. 

4.  When  you  initially  start  up  the  OPS  TEST  Administration  program, 
you  will  be  prompted  for  the  location  of  the  OPS  TEST  Backend 
Database.  Your  OPS  TEST  Administrator  will  provide  you  with  this 
path/location.  You  may  also  be  prompted  for  a  password  for  the 
OPS  TEST  Backend  Database.  This  also  can  be  provided  by  your 
OPS  TEST  Administrator. 

OPS  TEST  Support 

The  Institute  for  Information  Technology  Applications  (IITA)  can  provide 
some  key  support  to  assist  those  organizations  that  adopt  the  OPS  TEST 
application.  Specifically,  IITA  is  prepared  to  provide: 

■  Master  and  backup  compact  disks  that  include  the  setup  programs 
for  both  the  OPS  TEST  Administration  and  Test-Taking  software 
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components,  an  empty  database,  and  electronic  copies  of  both  the 
OPS  TEST  Administration  and  Test-Taking  manuals.  Upon 
request,  IITA  can  include  separate  versions  of  the  OPS  TEST 
Administration  components  for  the  training  and 
standardization/evaluation  functions  where  the  training  version  is 
precluded  from  accessing  tests  or  test  items  in  the  “Eval”  category. 

■  On-site  installation  and  initial  training  support  by  one  IITA  technical 
support  person.  This  support  would  typically  include  a  2-3  day  TDY 
funded  by  the  adopting  unit. 

■  Several  hardcopies  of  the  OPS  TEST  Administration  and  Test- 
Taking  user  manuals  together  with  PowerPoint  training  slides  and 
electronic  versions  of  the  same. 

■  Updated  version  releases  of  the  OPS  TEST  Administration  and 
Test-Taking  software  to  keep  OPS  TEST  current  with  subsequent 
releases  of  Microsoft  Office  (specifically  Word  and  Access). 
Software  releases  may  also  include  problem  resolutions  and 
functional  enhancements  possibly  including  items  identified  by  the 
adopting  unit.  However,  IITA  will  serve  as  the  final  software 
configuration  management  authority  and  will  make  all  decisions 
regarding  what  is  and  is  not  included  in  subsequent  software 
releases. 

■  Review  of  current  test  items  and  training  on  effective  test  item 
development.  This  support  would  also  be  funded  by  the  adopting 
unit  and  could  be  accomplished  in  conjunction  with  the  initial 
installation. 

■  Limited  telephone  technical  support  for  user  and  configuration 
issues  based  on  the  availability  of  IITA  technical  support  personnel. 
(However,  IITA  cannot  guarantee  such  availability). 

IITA  will  provide  the  support  mentioned  above  until  at  least  the  summer  of 
2001.  At  this  time,  IITA  will  evaluate  the  extent  of  OPS  TEST  usage  and 
formulate  continuing  support  plans  as  they  deem  appropriate. 

Pnoblem  Resolution  and  Suggestions 

If  you  find  problems  with  the  OPS  TEST  software  or  have  suggestions  to 
improve  the  application,  you  can  use  the  contact  information  below  to 
provide  the  details  of  your  problem  or  suggestion  to  the  IITA  Technical 
Support  Staff.  This  is  best  accomplished  via  an  email  or  postal  mail  to 
IITA.  You  should  provide  a  detailed  description  of  the  problem  or 
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suggestion,  version  of  OPS  TEST  that  you  are  running  (available  on  the 
main  menus),  description  of  the  computer  and  network  you  are  running  on, 
and  version  of  the  Windows  operating  system  you  are  using.  Also  provide 
contact  information  to  include  your  name,  phone,  and  email,  so  that  the 
technical  staff  can  get  further  information  as  needed. 

OPS  TEST  Points  of  Contact 

Further  information  and  support  on  OPS  TEST  can  be  obtained  from  the 
following  contacts  at  the  Institute  for  Information  Technology  Applications 
(IITA): 

■  Suggestions  &  Problem  Resolution:  Current  Managing  Director  of 
IITA,  (719)  333-6748,  DSN:  333-6748,  email:  iita@usafa.af.mil 

■  OPS  TEST  Project  Leader:  Dr.  Steve  Hadfield,  (719)  333-7474,  DSN: 
333-7474,  email:  steve.hadfield@usafa.af.mil 

■  Educational  Information  Technologies  Director:  Dr  Eric  Hamilton, 
(719)  333-8325,  DSN  333-8325,  email:  eric.hamilton@usafa.af.mil 

Or  via  mail  to:  I ITA,  HQ  USAFA/DFE,  Suite  4K25 

USAF  Academy,  CO  80840-6200 
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Chapter 


Test  Item  Maintenance 

How  do  you  add  new  and  update  existing  test  items? 

The  main  OPS  TEST  Administration  menu  (shown  in  Figure  1.1)  provides 
four  primary  functions  including  Test  Item  Maintenance,  Build/Maintain 
Tests,  View  Test  Results,  and  Test  Item  Analysis.  This  chapter 
discusses  the  first  of  these  functions  that  of  maintaining  test  items.  This 
includes  the  ability  to  selectively  access  and  update  existing  test  items  as 
well  as  to  add  new  test  items. 

Overview  of  Test  Item  Maintenance 

The  Test  Item  Maintenance  button  on  the  main  OPS  TEST 
Administration  menu  provides  access  to  a  query  form  from  which  you 
can  selectively  access  test  items  to  review  and/or  update.  You  can  also 
go  directly  to  add  new  test  items  from  this  form. 

Individual  test  items  are  shown  with  all  their  reference  information  on  a 
two-page  form.  On  the  first  page  are  the  actual  test  item  (held  as  a 
Word  document)  with  its  unique  Item  #  and  other  information  like  its 
associated  task  code,  classification,  dates  created  and  last  reviewed. 
Instructional  Systems  Development  (ISD)  proficiency/knowledge  level, 
number  of  choices,  correct  answer,  governing  regulation,  and  category. 
All  but  the  Item  #  can  be  changed  and  there  are  pull-down  lists  of 
currently  used  values  most  all  of  these  fields.  The  second  page 
provides  access  to  the  feedback  information  presented  to  test-takers 
when  they  incorrectly  answer  the  item,  some  Test  Administrator 
comments  on  the  test  item,  associated  multimedia  clips,  and  a  list  of 
centers  and  positions  that  this  test  item  is  associated  with.  Command 
buttons  on  this  form  allow  the  user  to  move  between  pages  of  the  form, 
mark  the  item  as  reviewed  as  of  the  current  day,  bring  up  a  summary  of 
how  the  test  item  has  performed  in  previous  usages,  find  another  test 
item  based  on  a  selected  field’s  value,  add  a  new  test  item,  save 
changes  to  the  current  test  item,  print  the  current  test  item  with  its 
reference  information,  un-do  the  last  set  of  changes  to  the  item,  delete 
the  test  item,  and  exit  the  form. 
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Selecting  Test  Itenns  for  Maintenance 

When  the  Test  Item  Maintenance  button  on  the  main  OPS  TEST 
Administration  menu  is  clicked,  the  query  form  found  in  Figure  3.1  is 
displayed  and  the  user  can  use  it  to  selectively  bring  up  the  test  items  they 
want  to  review  and/or  update.  Most  fields  on  this  form  have  pull-down  lists 
to  select  from.  You  can  also  go  directly  to  values  in  these  lists  by 
beginning  to  type  the  value  you  are  looking  for.  Furthermore,  some  of  the 
fields  allow  for  wildcard  selections  where  some  characters  are  specified  but 
“?”  and  “*”  can  also  be  included  to  represent  one  or  several  unspecified 
characters,  respectively.  For  example,  a  value  of  “NORAD*”  in  the 
References  field  would  bring  up  all  test  items  whose  associated  reference 
starts  with  “NORAD”  followed  by  whatever. 

In  general,  all  criteria  specified  on  this  selection  form  must  be  satisfied  by 
the  test  items  to  be  returned.  The  one  exception  to  this  rule  is  Center  and 
Position.  Since  positions  are  typically  tied  to  a  center,  specification  of  both 
a  Center  and  a  Position  (from  a  different  center)  will  return  test  items 
associated  with  the  center  OR  with  the  position.  If  you  do  not  have 
evaluator  privileges  (the  standardization/evaluation  version  of  OPS  TEST 
Administration),  you  will  automatically  be  excluded  from  accessing  any  test 
items  whose  Category  starts  with  “Eval”,  that  is,  you  can  not  access 
evaluation  questions.  To  get  details  on  the  usage  and  format  of  the  fields 
in  the  query  form,  simply  place  your  cursor  over  the  field  and  a  caption  box 
will  appear  with  a  description  of  that  field. 


Figure  3.1 :  Select  Test  Items  for  Maintenance  Form 
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Test  Item  Maintenance  Form 

Once  any  desired  selection  criteria  has  been  specified,  click  on  the 
Retrieve  Selected  Test  Items  button  to  bring  up  the  test  items.  If  you 
wish  to  add  new  test  items  to  the  Testbank,  click  on  the  Add  New  Test 
Items  button  which  will  disregard  any  specified  selection  criteria  and  bring 
up  a  blank  copy  of  the  Test  Item  Maintenance  form. 

Existing  test  items  selected  for  review/update  and  new  items  to  be  added 
to  the  Testbank  are  shown  via  the  two-page  Test  Item  Maintenance  form 
shown  in  Figures  3.2  and  3.3.  The  fields  on  this  form  are  labeled  and 
fairlyself-explanatory. 


Figure  3.2:  Test  Item  Maintenance  Form 

The  first  is  Item  #  which  is  a  unique  numeric  identifier  for  each  test  item  that 
is  automatically  assigned  by  the  system  and  can  not  be  changed  by  the 
user.  The  other  fields  are: 

•  Task:  Task  code  that  identifies  the  area  addressed  by  this  test  item. 
The  associated  pull-down  list  has  a  complete  listing  of  all  task  codes 
that  have  been  set  in  the  configuration  of  the  system. 
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•  Classification:  Security  classification  of  the  test  item.  The  pull-down 
list  shows  the  allowed  values. 

•  Created:  The  date  the  test  item  was  created  (or  at  least  when  it  was 
added  to  the  Testbank). 

•  Reviewed:  The  date  the  test  item  was  last  reviewed  and  validated  by  a 
Testbank  Administrator. 

•  Proficiency:  The  Instructional  Systems  Development  (ISD) 

proficiency/knowledge  level  with  values  A-D  indicates  the  level  at  which 
the  item  tests  the  topic. 

•  #  Choices:  The  number  of  choices  provided  for  the  test  item  (a 
multiple  choice  format  is  assumed  of  all  test  items).  The  value  in  this 
field  is  used  for  subsequent  test-taking  to  determine  how  many  choices 
to  enable  for  this  test  item.  Thus  it  is  important  that  this  value  match  the 
number  of  options  in  the  actual  test  item.  There  must  be  at  least  two 
choices  and  no  more  than  five. 

•  Correct  Answer:  Correct  answer  for  the  test  item  with  the  assumption 
that  the  choices  are  labeled  with  letters  of  the  alphabet. 

•  Governing  Regulation,  Operating  Instruction,  Other  Guidance: 

The  authoritative  reference  for  this  test  item.  This  information  will  be 
displayed  to  the  test-taker  as  feedback  if  they  incorrectly  answer  the 
test  item. 

•  Category:  Identifies  the  type  of  test  item  this  is.  Some  special  values 
in  this  fieid  are  keyed  in  on  by  OPS  TEST.  Specificaily,  values  that 
start  with  “Evai”  are  used  to  identify  evaluation  test  items  to  which 
those  without  evaluator  privileges  cannot  get  access.  Values  that 
start  with  “MQF”  are  used  to  identify  Master  Question  Fiie  test 
items  that  may  be  accessed  by  anyone  for  taking  practice  tests. 

•  Dormant?:  Is  a  yes/no  box  that,  if  checked,  means  the  test  item  is  no 
longer  in  use. 

•  Actual  Test  Item:  The  actual  test  item  is  held  as  a  Word  document  in 
the  large  box  in  the  center  of  the  form.  You  can  double  click  within  this 
box  to  activate  Word  operating  on  the  actual  test  item.  In  this  way,  you 
can  update  the  test  item  using  all  the  capabilities  of  Microsoft  Word. 
When  adding  a  new  test  item,  this  box  will  be  empty  and  a  double-click 
inside  of  it  will  cause  a  test  item  template  to  be  copied  in  and  Word  to 
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be  activated  so  you  can  type  in  the  new  test  item.  As  these  test  items 
are  Word  documents,  they  can  hold  any  images,  equations,  or  graphics 
that  can  normally  be  included  within  Word  documents. 

On  the  second  page  of  the  Test  Item  Maintenance  form  (shown  in  Figure 

3.3),  there  are  the  following  fields: 

•  Feedback  to  Test-Takers:  Text,  in  open  memo  format,  that  will  be 
displayed  to  the  test-taker  if  they  incorrectly  answer  the  test  item. 

•  Hyperlink  Ref:  This  field  can  be  used  to  specify  a  link  to  a  file  or  URL 
(Universal  Resource  Locator)  that  contains  additional  information 
pertaining  to  the  test  item  that  is  provided  to  the  test-taker  if  they 
incorrectly  answer  the  item. 

•  Test  Administrator  Comments:  Any  comments  that  the  Testbank 
Administrators  would  like  carried  with  the  test  item.  These  are  only 
visible  to  the  Testbank  Administrators  and  are  to  be  used  in  whatever 
way  they  see  fit  (or  not  at  all). 


Figure  3.3:  Test  Item  Maintenance  Form  (Page  2) 
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•  Associated  Centers/Positions:  A  list  of  the  center/crew  positions  that 
are  associated  with  this  test  item.  There  can  be  any  number  of  such 
center/positions  in  the  list  and  pull-down  lists  let  the  user  select  from 
valid  options  as  specified  in  the  system  configuration.  The  Delete 
button  allows  center/position  pairs  to  be  removed  from  the  list  and  the 
reversed  arrow  button  allows  the  latest  change  to  be  “un-done”. 

•  Multimedia:  Is  an  Object  Linking  and  Embedding  (OLE)  field  that  can 
contain  any  type  of  multimedia  file  that  has  an  application  to  support  it 
on  the  host  systems.  These  can  be  video  clips,  static  images,  or  sound 
clips.  The  Paste  Multimedia  from  File  and  Paste  Multimedia  from 
Clipboard  buttons  below  it  can  be  used  to  add  such  multimedia  to  the 
test  item.  When  test  items  are  displayed  that  include  associated 
multimedia,  the  Play  Multimedia  button  will  appear  on  the  test-taking 
form.  Clicking  this  button  will  activate  the  multimedia  clip  and  it’s 
application  per  the  association  established  on  that  computer.  NOTE:  If 
you  are  going  to  use  multimedia  in  a  test  item,  you’ll  want  to  be  sure 
that  all  machines  that  will  be  used  to  answer  that  test  item  have  an 
appropriate  application  associated  with  the  file  extension  for  that  type  of 
multimedia. 

•  Included  in  Master  Tests:  A  list  of  the  master  tests  that  use  this  test 
item  is  provided  for  reference  to  help  identify  the  impacted  tests  if  this 
test  item  is  altered  or  deleted.  Master  tests  will  use  the  latest  update  of 
each  test  item  and  deleted  items  will  be  automatically  removed  from 
master  tests. 

•  #  Times  used  in  Specific  Tests:  The  number  of  times  this  test  item  is 
used  in  specific  tests  (either  taken  or  awaiting  to  be  taken)  is  included  to 
help  identify  the  impacts  of  altering  or  deleting  the  test  item.  Un-taken 
specific  tests  will  use  the  latest  updated  version  of  the  test  item  and 
deleting  the  item  will  remove  it  from  these  tests. 

At  the  bottom  of  both  pages  of  the  Test  Item  Maintenance  form  are  sets  of 
command  buttons  for  use  by  the  Testbank  Administrators.  The  functions  of 
these  buttons  are  as  follows  (you  can  also  get  a  description  of  the  button’s 
function  by  placing  your  cursor  over  it  or  tabbing  to  it  which  causes  a 
caption  box  to  appear): 

•  Move  to  Other  Page:  The  leftmost  button  with  the  pointed  hand  on  it 
moves  the  user  between  the  two  pages  of  the  Test  Item  Maintenance 
Form. 
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•  Checkmark:  The  checkmark  button  has  the  date  last  Reviewed  for  the 
test  item  set  to  the  current  date  and  is  intended  to  help  Testbank 
Administrators  efficiently  do  their  periodic  reviews  of  the  test  items. 

•  Chart:  The  chart  button  shows  an  analysis  of  the  past  performance  of 
the  currently  displayed  test  item.  More  details  on  the  information 
provided  from  this  analysis  can  be  found  in  Chapter  6. 

•  Binoculars:  The  binoculars  button  allows  the  user  to  do  a  “find” 
operation  based  on  whatever  test  item  field  was  currently  selected  prior 
to  clicking  on  this  button.  For  example,  to  find  a  test  item  in  the 
currently  selected  set  that  has  a  Proficiency  value  of  “A”,  click  on  the 
Proficiency  field  and  then  click  the  binoculars  button.  A  dialog  box  will 
be  displayed  into  which  you  type  an  “A”  and  then  click  on  the  “Find 
First”  or  “Find  Next”  buttons.  You  will  then  be  taken  to  the  appropriate 
test  item  satisfying  that  criteria. 

•  Add  New:  The  add  new  button  has  a  right  arrow  followed  by  a  star. 
Clicking  on  this  button,  saves  any  changes  to  the  current  test  item  and 
then  blanks  out  the  form  so  a  new  test  item  may  be  added. 

•  Save:  The  button  with  a  diskette  and  an  arrow  on  it  is  used  to  save  the 
current  test  item  and  any  specified  updates.  Saves  are  also 
automatically  done  by  moving  between  test  items  and  closing  the  form. 

•  Print:  The  button  with  a  printer  on  it  is  used  to  print  the  current  test 
item  and  its  reference  information  to  the  default  system  printer. 

•  Undo:  The  undo  button  has  a  counter-clockwise  arrow  on  it.  This 
button  un-does  the  latest  changes  to  the  test  item.  It  is  also  quite  useful 
when  you  start  adding  a  new  item  and  then  decide  not  to  proceed.  If  all 
the  required  fields  have  not  been  specified,  you  can  be  stuck  not  able  to 
save  or  exit  the  form.  Clicking  the  undo  button  will  clear  the  partially 
added  item  and  let  you  exit  the  form. 

•  Delete:  The  delete  button  has  a  trash  can  on  it  and  is  used  to 
permanently  delete  the  currently  displayed  test  item.  NOTE:  This 
button  must  be  used  with  caution  as  it  also  removes  the  test  item  from 
any  tests  that  it  has  been  included  in  as  well  as  from  the  results  of 
previously  taken  tests.  A  better  alternative  to  deleting  the  item  is  to 
mark  the  test  item  as  dormant  which  retains  the  item  and  its  usage  in 
any  tests  but  precludes  it  from  normal  searches. 
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•  Close  Form:  The  close  form  button  has  an  open  door  on  it  and  allows 
the  user  to  close  the  Test  Item  Maintenance  form.  This  also  causes 
any  changes  to  the  currently  displayed  test  item  to  be  saved. 

At  the  bottom  of  the  Test  Item  Maintenance  form  is  a  navigation  bar  with 
arrows  that  allow  movement  within  the  current  set  of  selected  test  items.  It 
also  shows  the  number  of  your  currently  displayed  test  item  (relative  to  the 
selected  set)  as  well  as  the  total  number  of  selected  test  items. 
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Chapter 


Build  and  Maintain  Tests 

How  do  you  build  and  mainlain  boti  master  tests  and 
copies  of  master  tests  for  specific  test-takers? 


The  second  button  on  the  main  OPS  TEST  Administrator  menu  (Figure 
1.1)  provides  access  to  the  mechanisms  for  building  and  maintaining 
master  tests  as  well  as  to  the  handling  of  specific  tests  which  are  built  from 
the  master  tests  for  individual  crew  members  needing  to  take  that  test. 

Build/Maintain  Tests  Menu  Form 

When  the  Build/Maintain  Tests  button  is  clicked  off  the  main  OPS  TEST 
Administrator  menu,  the  Build/Maintain  Tests  form  (see  Figure  4.1)  is 
displayed.  There  are  two  main  areas  to  this  form  corresponding  to  Master 
Tests  and  those  Specific  Tests  made  from  the  Master  Tests  for  specific 
crew  members. 


Figure  4.1  Build/Maintain  Tests  Form 
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Dealing  with  Master  Tests 

The  concept  of  operations  for  the  OPS  TEST  is  to  make  one  or  two  Master 
Tests  for  each  crew  position  and  then  to  derive  specific  instances  of  the 
master  test  for  individual  crew  members  needing  to  test  for  that  position. 
The  four  buttons  on  the  left  side  of  the  Build/Maintain  Tests  form  provide 
means  to  deal  with  Master  Tests.  These  buttons  are: 

•  Create  New  Master  Test:  This  button  opens  up  a  blank  copy  of  the 
Master  Test  form  (to  be  discussed  shortly)  which  can  be  used  to  build 
an  entirely  new  Master  Test. 

•  Make  Copy  of  Master  Test:  Typically  Master  Tests  are  not  completely 
rewritten  each  year.  More  often  the  old  test  is  modified  to  become  the 
next  year’s  test.  To  facilitate  this  practice,  the  Make  Copy  of  Master 
Test  button  allows  an  existing  Master  Test  to  be  copied  and  given  a 
new  name.  This  new  copy  of  the  Master  Test  can  then  be  updated,  via 
the  Review/Update  Master  Test  button,  to  become  the  new  test  for  the 
coming  year.  When  this  button  is  clicked,  a  dialog  form  comes  up  that 
allows  you  to  select  an  existing  Master  Test  to  copy  and  to  specify  a 
name  for  the  copy  to  be  made.  The  software  automatically  checks  that 
you  do  not  create  name  conflicts  by  doing  this. 

•  Review/Update  Master  Test:  You  must  first  select  an  existing  Master 
Test  in  the  combo  box  below  and  then  click  on  this  button  to  review  and 
update  that  Master  Test.  The  review  and  update  process  is  facilitated 
by  the  Master  Test  form  to  be  discussed  shortly.  If  there  are  any 
Specific  Tests  made,  but  not  yet  taken,  from  the  selected  Master  Test, 
the  software  will  warn  you.  Any  changes  made  to  the  Master  Test  will 
not  be  reflected  in  these  pending  Specific  Tests.  You  can,  however, 
delete  the  pending  Specific  Tests  and  recreate  them  after  you  have 
updated  the  Master  Test. 

•  Delete  Master  Test:  To  delete  an  existing  Master  Test,  you  must  first 
select  it  in  the  combo  box  provided  below  the  button.  The  deletion 
process  will  check  for  pending  Specific  Tests  made  from  the  selected 
Master  Test  but  not  yet  taken.  These  Specific  Tests,  and  the  results  of 
any  previously  taken  instance  of  this  Master  Test,  will  not  be  deleted 
when  you  delete  the  Master  Test. 

Dealing  with  Specific  Tests 

Specific  Tests  are  copies  of  a  Master  Test  to  be  taken  by  a  specific 
individual  crew  member.  The  buttons  on  the  right  side  of  the 
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Build/Maintain  Tests  form  allow  you  to  deal  with  Specific  Tests.  There 

are  three  such  buttons  and  they  are: 

•  Make  New  Specific  Test  from  Master:  This  button  opens  up  a 
blank  copy  of  the  Specific  Tests  form  (to  be  discussed  shortly), 
which  can  then  be  used  to  build  a  specific  instance  of  a  Master  Test 
for  an  individual  crew  member. 

•  Review/Reset/Reorder  Specific  Test:  To  use  this  button,  you 
must  first  select  an  existing  Specific  Test  from  the  combo  box  below. 
The  button  will  cause  this  Specific  Test  to  be  displayed  in  the 
Specific  Tests  form  from  where  it  can  be  reviewed,  reset  (so 
interrupted  tests  can  be  re-entered  by  the  test-taker),  and  reordered. 
If  the  selected  Specific  Test  was  already  taken,  you  can  use  this 
form  to  review  and  print  the  results. 

•  Delete  an  Existing  Specific  Test:  First  select  an  existing  Specific 
Test  in  the  combo  box  below  and  then  click  this  button  to  have  the 
test  deleted.  NOTE:  Deleting  an  already  taken  Specific  Test  will 
discard  all  record  of  how  the  included  test  items  performed  on  that 
Specific  Test. 

Master  Test  Fomn 

The  creation,  review,  and  update  of  Master  Tests  is  done  via  the  Master 

Test  Form  shown  in  Figure  4.2.  The  fields  included  on  this  form  include: 
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Master  Tests 


Master  test  [Missile  warning  Center  3]  Position  [mwocdr  3  Category  Z] 
Type  of  test  [special  3  OPit  I^^S  ZJ  Originator  [Hadtield,  Steven  M.  3 


Make  on  request?  ^ ^ ^  jg jj  ugg, 

I  Random  order?  - 

l~  Re-entrant? 

Password;  [  PPiot  Test  Summary 


Reorder  Test  Items  Preview  Test  On-Line 


Print  tull  Test 


tn 


Close/Save 


Figure  4.2:  Master  Test  Form 


•  Master  Test  Name:  A  20-character  identifier  of  the  Master  Test  that 
typically  includes  reference  to  the  Center,  Crew  Position(s),  and 
timeframe  of  its  use. 

•  Center:  The  operations  center  that  this  Master  Test  is  associated  with. 

•  Position:  The  Crew  Position  that  this  test  is  associated  with.  If  there 
are  multiple  positions,  you  can  just  list  one  or  none  of  them. 

•  Category:  The  Category  of  the  test  is  used  to  identify  evaluation  tests 
(by  specifying  “Eval”  followed  by  whatever)  and  precluding  access  to 
these  tests  by  Testbank  Administrators  that  do  not  have  evaluator 
privileges.  For  non-evaluation  test,  you  may  put  whatever  you  like  for 
category. 

•  Type  of  Test:  Typically  tests  are  classified  as  “Initial”,  “Recurring”,  and 
“Special”,  but  other  values  are  also  allowed  in  this  field. 

•  OPR:  The  Office  of  Primary  Responsibility  for  the  test.  Typically,  this  is 
“stan/eval”  for  Eval  tests  and  “training”  for  training  tests,  but  you  can 
specify  whatever  you  wish  here. 
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•  Originator:  Person  originating  this  Master  Test. 

•  #:  A  sequence  number  given  to  each  test  item  in  the  Master  Test 
starting  with  one  and  going  through  the  number  of  items  in  the  test. 
You  can  change  the  numbers  in  these  fields  for  all  test  items  currently 
in  the  Master  Test  and  then  click  on  the  Reorder  Test  Items  button 
which  will  sort  the  test  items  per  the  numbers  you  assigned  and  then 
renumber  the  test  items  per  the  new  ordering.  NOTE:  You  may  skip 
numbers  in  this  reordering  process  and  the  system  wiii  automaticaiiy 
provide  a  correct  ordering  eiiminating  any  skipped  vaiues.  You  can 
aiso  use  negative  numbers  to  move  items  to  the  beginning  of  the  test. 

•  Item  #:  The  unique  Testbank  Item  #  for  each  test  item  included  in  the 
Master  Test. 

•  Task:  The  task  code  for  each  test  item  included  in  the  Master  Test. 

•  Reference:  The  governing  regulation,  operating  instruction  or  other 
guidance  that  provides  the  authority  for  each  included  test  item. 

•  Answer:  The  correct  answer  for  each  included  test  item. 

•  Make  on  request?  Checking  this  box  indicates  that  specific  instances 
of  this  test  do  not  need  to  be  created  separately.  They  will  be  make 
upon  request  via  the  Take  Monthly  training  Test  button  from  within  OPS 
TEST  Test-Taking.  Selecting  this  option  also  enables  the  three  fields 
below  it;  Random  order?,  Re-entrant”,  and  Password. 

•  Random  order?;  This  is  an  option  for  “Make  on  request?”  (monthly 
training)  tests.  Checking  it  specifies  that  the  items  in  the  test  will  be 
randomly  reordered  when  each  instance  of  the  test  is  built. 

•  Re-entrant?:  This  is  another  option  for  the  “Make  on  request?” 
(monthly  training)  tests.  Checking  it  specifies  that  the  test-taker  will  be 
allowed  to  enter  and  work  on  the  test,  then  exit  without  having  it  graded 
and  re-enter  it  at  a  later  time.  All  previous  responses  will  be  preserved. 

•  Password:  This  is  an  optional  parameter  for  “Make  on  request?” 
(monthly  training)  tests.  The  text  (letters,  numbers,  and/or  symbols)  will 
be  the  global  password  for  creating  and  taking  instances  of  this  test.  If 
left  blank,  the  test  will  not  be  password  protected. 

The  Master  Test  form  also  has  several  command  buttons  for  use  in  dealing 

with  Master  Tests.  For  each  included  test  item,  there  are  the  following 
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buttons  (you  must  first  click  on  that  test  item  and  then  click  on  it’s  specific 

buttons): 

•  View:  Allows  you  to  bring  up  that  test  item  in  a  form  very  similar  to  the 
Test  Item  Maintenance  form  so  you  can  look  at  the  item  and  its 
associated  reference  information. 

•  Delete:  Allows  you  to  delete  this  test  item  from  the  Master  Test. 

•  Undo:  The  button  with  the  reversed  arrow  undoes  (removes)  any 
changes  you  may  have  inadvertently  made  within  the  entry  for  this  test 
item. 

The  remaining  buttons  on  the  Master  Test  form  pertain  to  the  test  as  a 

whole.  They  are: 

•  Add  New  Test  Item:  This  button  brings  up  a  query  form  for  you  to  use 
to  select  test  items  to  consider  for  inclusion  in  the  Master  Test.  The 
process  of  adding  new  test  items  to  a  Master  Test  is  discussed  in 
further  detail  below. 

•  Reorder  Test  Items:  To  use  this  button,  first  specify  the  relative 
ordering  of  test  items  currently  in  the  Master  Test  by  typing  these 
numbers  in  the  #  field  for  each  test  item.  Then  click  on  the  Reorder 
Test  Items  button  to  have  the  test  items  sorted  by  your  new  numbering 
and  then  renumbered  according.  You  can  use  numbering  gaps, 
negative  numbers  (to  move  items  to  the  beginning)  and  very  large 
numbers  (to  move  items  to  the  end). 

•  Preview  Test  On-Line:  This  button  allows  you  to  see  what  the  test  will 
look  like  as  seen  by  the  test-taker  and  is  discussed  in  more  detail 
shortly. 

•  Print  Test  Summary:  Causes  a  summary  listing  of  the  test  items 
included  in  the  Master  Test  to  be  printed  to  the  default  printer. 

•  Print  Fuli  Test:  Produces  a  full  print  out  of  the  Master  Test  (to  include 
the  actual  test  items  themselves)  to  be  printed  on  the  default  printer. 

•  Close  and  Save:  Saves  any  changes  to  the  Master  Test  and  closes 
the  Master  Test  form. 
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Specifying  Master  Test  Information 

Most  of  the  information  on  the  Master  Test  form  can  be  specified  using 
pull-down  lists  associated  with  the  fields.  The  key  exceptions  are  the  name 
of  the  Master  Test,  which  is  a  20-character  field  that  usually  includes  the 
operations  center,  positions,  and  time-frame,  and  the  included  test  items 
that  are  actually  added  via  the  Add  New  Test  Item  button. 

Adding  New  Test  Items  to  a  Master  Test 

New  test  items  must  be  added  to  the  Master  Test  via  the  Add  New  Test 
Item  button  on  the  Master  Test  form.  Clicking  this  button  brings  up  a  query 
form  to  be  used  to  retrieve  test  items  for  consideration  to  be  included  in  the 
Master  Test.  This  query  form  is  shown  as  Figure  4.3  below.  Once  you 
specify  the  selection  criteria,  click  on  the  Retrieve  Selected  Test  Items 
button  to  obtain  those  test  items  that  meet  all  of  the  specified  criteria.  (The 
only  exception  to  this  rule  is  if  both  Center  and  Position  are  specified  and 
they  reference  different  centers,  items  matching  either  the  Center  or 
Position  criteria  will  be  included).  Wildcards  are  allowed  in  some  fields.  To 
get  the  specifics  on  each  field,  move  your  cursor  over  the  field  and  a 
caption  box  will  appear  and  provide  you  the  details. 


Figure  4.3:  Selecting  Test  Items  for  a  Master  Test 

The  test  items  retrieved  via  the  Select  Test  Items  for  a  Master  Test  form 
are  displayed  in  a  form  very  similar  to  the  Test  Item  Maintenance  form  (see 
Figures  3.2  and  3.3).  There  is  a  new  button  on  this  form  that  has  a  right 
pointing  arrow  to  a  little  document  icon  in  the  lower  right  corner  of  the  form. 
Clicking  on  this  button  will  include  the  currently  displayed  test  item  into  the 
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Master  Test  (unless  that  test  item  is  already  in  the  Master  Test,  in  which 
case  an  error  message  is  displayed). 

Viewing  and  Deleting  Test  Items  from  a  Master  Test 

Viewing  and  deleting  test  items  from  a  Master  Test  is  done  via  the  small 
buttons  provided  with  each  included  test  item.  The  View  button  brings  up 
that  test  item  in  a  form  very  similar  to  the  Test  Item  Maintenance  form  (see 
Figures  3.2  and  3.3).  The  Delete  button  removes  that  test  item  from  the 
Master  Test. 

Reordering  Test  Items  Within  a  Master  Test 

You  can  re-order  the  test  items  currently  in  a  Master  Test  by: 

1 .  Specifying  new  #  field  entries  for  the  test  items  currently  in  the  Master 
Test  in  accordance  with  where  you  would  like  them  to  be  in  the  test. 
Gaps  in  the  numbering  are  no  problem  as  the  software  will  later  sort 
and  renumber  the  test  items  per  your  specified  ordering.  You  can  also 
use  negative/large  positive  numbers  to  move  items  to  the  beginning  or 
end  of  the  test  respectively. 

2.  Click  the  Reorder  Test  Items  button  which  will  cause  the  system  to 
sort  the  test  items  per  your  specified  ordering  and  then  sequentially 
renumber  them  appropriately  (thus  getting  rid  of  any  gaps  you  may 
have  left). 

Previewing  a  Master  Test 

Clicking  on  the  Preview  Test  On-Line  button  lets  you  see  how  the  Master 
Test  will  look  to  the  test-taker  when  taken  on-line  (see  Figure  4.4  below). 
The  display  is  slightly  different  from  the  test-taker’s  view  in  that  it  includes 
the  Testbank  Item  #  of  each  test  item  and  the  feedback  information  for 
each  test  item.  If  the  test  item  has  associated  multimedia,  the  Play 
Multimedia  button  is  provided  and  clicking  it  will  play  the  associated 
multimedia  clip  (as  shown  in  Figure  4.4).  If  a  hyperlink  is  associated  with 
the  test  item,  it  is  also  provided  on  the  display  and  is  active  so  you  can  click 
on  it  to  see  the  referenced  material.  Also  the  correct  answer  to  the  test 
item  is  displayed  in  the  Answer  Here  selection  box.  These  are  provided 
so  you  can  easily  review  and  evaluate  your  Master  Test. 
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Figure  4.4:  Master  Test  Preview  Form 
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Specific  Test  Form 


When  creating  or  reviewing  Specific  Tests  via  the  right  side  buttons  on  the 
Build/Maintain  Tests  form  (see  Figure  4.1),  the  Specific  Tests  are  shown  in 
the  Specific  Tests  for  Crew  Members  form  (shown  as  Figure  4.5).  On  this 
form,  there  are  the  following  fields: 


Figure  4.5:  Specific  Test  Form 

•  Master  Test:  Name  of  the  Master  Test  from  which  this  Specific  Test  is 
derived.  Should  be  selected  from  the  pull-down  list. 

•  Test-taker:  Name,  grade,  and  social  security  number  of  the  crew 
member  who  is  to  take  this  test. 

•  Password:  To  protect  this  Specific  Test  from  access  by  those  other 
than  for  whom  it  was  built,  there  is  a  password  assigned.  The  system 
will  automatically  default  this  to  a  random  value,  but  this  value  can  be 
overwritten  by  the  Testbank  Administrator. 

•  Position:  The  center/crew  position  for  which  the  test-taker  is  to  take 
this  test. 


26 


OPS  TEST  ADMINISTRATOR’S  MANUAL 


•  Can  Re-Enter?:  If  checked,  allows  the  Test-Taker  to  close  the  test¬ 
taking  form  and  later  re-enter  the  test.  If  un-checked,  such  re-entry  is 
not  allowed.  In  no  case  is  a  Test-Taker  allowed  to  re-enter  a  test  that 
has  been  graded  (without  direct  intervention  from  the  test  administrator 
to  “reset”  the  test). 

•  Date  Created:  Is  the  date  this  Specific  Test  was  built.  This  date  is 
defaulted  to  the  current  day  and  is  also  displayed  when  the  test-taker 
goes  to  access  their  test  per  the  instructions  in  Chapter  7. 

•  Retest?:  Indicates  if  this  is  a  retest  (if  checked). 

•  Suspense:  Optional  suspense  date  by  which  this  Specific  Test  must 
be  taken.  This  date  will  show  up  in  the  Test  Results/Status  summary 
described  in  Chapter  5. 

•  Administered  By:  Indicates  who  is  building/administering  this  Specific 
Test. 

•  Start  Date/Time:  Set  when  the  test-taker  starts  the  test.  Once  set,  the 
test-taker  can  not  exit  the  test  and  re-enter  it.  A  Test  Administrator 
must  first  use  the  Reset  Test  button  to  clear  this  date  and  any  test- 
taker  responses  for  the  test-taker  to  get  back  into  the  test. 

•  Finish  Date/Time:  When  the  test-taker  finished  the  test  by  clicking  on 
the  Grade  Test  button. 

•  Score:  The  percentage  of  test  items  the  test-taker  correctly  answered, 
if  the  test  was  already  taken  and  graded. 

•  Results  Posted?:  Indicates  that  the  results  of  this  Specific  Test  have 
been  posted  to  the  appropriate  personnel  records  (if  checked). 

•  Random  Order?:  Check  box  that  indicates  whether  this  Specific  Test 
has  been  randomly  reordered  (if  checked)  or  if  it  retains  the  original 
ordering  of  test  items  from  the  Master  Test.  The  Random  Order  button 
toggles  between  these  two  options. 

The  Specific  Tests  for  Crew  Members  form  in  Figure  4.5  has  several 

buttons  for  use  in  dealing  with  Specific  Tests.  These  buttons  are: 

•  Preview  On-Line:  Provide  a  preview  of  the  test  items  in  the  Specific 
Tests.  In  this  preview  format,  any  test-taker  answers  will  be  shown  in 
the  Answer  Here  selection  box.  You  can  also  provide  test-taker 
responses  on  this  form  which  allows  you  to  specify  responses  given  by 
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test-takers  on  a  hardcopy  of  the  test.  Other  reference  information  is 
also  provided  in  this  view  (that  regular  test-takers  don’t  see)  to  help  you 
in  reviewing  the  test.  NOTE:  You  can  also  update  the  answers  if 
necessary  and  hit  the  Grade  Test  button  to  have  it  regraded. 

•  Print  Hardcopy:  Provides  a  printed  version  of  the  test  suitable  for 
administering  the  test  off-line. 

•  Print  Answer  Key:  Prints  the  answer  key  for  the  Specific  Test  to  the 
default  printer. 

•  Random  Order:  Toggles  between  a  random  ordering  of  the  test  items 
in  the  Specific  Test  and  the  original  order  from  the  associated  Master 
Test.  The  Random  Order?  check  box  on  the  form  shows  the  current 
state. 

•  Reset  Test:  Clears  the  Start  and  Finish  Date/Times,  the  score,  and  all 
test-taker  responses,  so  the  test  can  be  “re-taken”  even  if  it  had  already 
been  graded. 

•  Print  Results:  Provides  a  summary  report  of  the  results  of  this  Specific 
Test  including  test-taker  responses  to  each  test  item. 

•  Undo:  Un-does  any  currently  pending  changes  to  the  Specific  Test. 

•  Save:  Saves  any  changes  to  the  Specific  Test.  (The  Close  Form 
button  also  causes  any  pending  changes  to  be  saved). 

•  Close  Form:  Causes  any  pending  changes  to  the  Specific  Test  to  be 
saved  and  the  Specific  Tests  form  to  be  closed. 

Making  Specific  Tests  (from  a  Master  Test) 

To  make  a  Specific  Test  from  a  Master  Test: 

1.  Click  on  the  Make  New  Specific  Test  from  Master  button  on  the 
Build/Maintain  Tests  form. 

2.  Complete  the  Specific  Tests  for  a  Crew  Member  form  per  the  directions 
and  descriptions  above. 

3.  Use  the  Print  Hardcopy  button  to  print  a  version  of  the  test  to  take  off¬ 
line.  (If  this  Specific  Test  is  to  be  taken  on-line,  skip  this  step). 

4.  Save  the  Specific  Test  with  either  the  Save  or  Close  Form  buttons. 
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Password  Protection  on  Specific  Tests 

A  password  mechanism  is  provided  for  all  Specific  Tests  to  preclude 
access  by  those  other  than  who  the  test  was  designed  for.  This  password 
is  defaulted  to  a  random  value  on  the  Specific  Tests  for  Crew  Members 
form,  but  it  can  be  over-written  by  typing  in  a  new  password  on  that  form. 
The  password  then  provides  a  controlled  access  to  the  Specific  Test. 

Establishing  an  Ordering  of  Test  Items 

Specific  Tests  may  be  either  randomly  ordered  or  retain  the  original 
ordering  of  test  items  from  the  Master  Test.  The  Reorder  Test  button  on 
the  Specific  Tests  for  Crew  Members  form  toggles  between  these  two 
options  and  the  Random  Order?  check  box  on  that  form  gives  the  current 
state  of  the  test. 

Resetting  a  Specific  Test 

Once  a  Specific  Test  has  been  started  and  does  not  have  the  “Can  Re- 
Enter?”  box  checked,  it  cannot  be  exited  and  re-entered  without 
intervention  from  a  Test  Administrator.  If  the  “Can  Re-Enter?”  box  is 
checked,  they  can  get  back  in  but  only  if  the  test  has  not  yet  been  graded. 
If  in  either  of  these  cases,  the  test-taker  has  been  locked  out  and  needs  to 
re-enter  the  test,  the  test  can  be  reset  by  the  test  administrator. 

If  a  currently  locked  out  Specific  Test  needs  to  be  “re-entered”,  the  Test 
Administrator  can  call  up  the  Specific  Test  via  the  Review/Reset/Reorder 
Specific  Test  button  on  the  Build/Maintain  Tests  form.  Then,  use  the 
Reset  Test  button  on  the  Specific  Tests  for  Crew  Members  form  to  clear 
the  Start  and  Finish  Date/Times,  the  Score,  and  all  responses  to  test  items 
on  that  Specific  Test  at  which  point  the  Specific  Test  can  be  re-entered  and 
taken. 

Previewing/Reviewing  a  Specific  Test 

The  Specific  Test  can  be  previewed  (or  reviewed  if  already  taken)  via  the 
Preview  On-Line  button  on  the  Specific  Tests  for  Crew  Members  form. 
When  this  is  done,  each  test  item  in  the  Specific  Test  will  be  shown  using 
the  form  shown  in  Figure  4.6  below.  On  this  form,  any  answers  from  the 
test-taker  will  be  shown  in  the  Answer  Here  selection  box.  These 
responses  can  also  be  specified  at  this  time,  which  allows  the  Test 
Administrator  to  input  the  responses  from  a  Specific  Test  taken  off-line. 
The  Grade  Test  button  will  calculate  and  record  the  grade  of  this  test  for 
you  if  you’re  having  to  input  the  test-taker’s  responses  yourself. 
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If  you  ever  encounter  a  question  that  has  the  wrong  correct  answer 
specified  and  the  test-taker  answers  with  the  real  correct  answer,  you  can 
review  the  test  to  change  their  answer  to  that  which  would  give  them  credit 
for  the  test  item  and  click  the  Grade  Test  button  to  have  their  score 
updated  appropriately. 

On  this  Specific  Test  Preview  form,  you  can  move  through  the  test  using 
either  set  of  arrow  buttons.  The  left  set  (with  the  shadows)  moves  to  the 
previous/next  un-answered  test  item.  The  right  set  moves  to  the 
previous/next  test  item  regardless  of  whether  or  not  it  has  been  answered. 

Printing  Hardcopies  of  Specific  Tests  and  Results 

You  can  print  a  hardcopy  of  a  Specific  Test  suitable  for  taking  the  test  off¬ 
line  by  calling  up  the  Specific  Test  via  the  Review/Reset/Reorder  Specific 
Test  button  on  the  Build/Maintain  Tests  form.  Then  use  the  Print 
Hardcopy  button  to  obtain  the  print  out.  You  can  get  an  answer  key 
printed  via  the  Print  Answer  Key  button.  You  can  also  print  a  summary  of 
a  taken  Specific  Test  by  the  Print  Results  button. 


Figure  4.6:  Specific  Test  Preview  Form 
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Chapter 


Viewing  Test  Results 

How  do  you  check  on  the  results  of  taken  tests  and  the 
status  ofun-taken  tests? 


Test  administrators  can  quickly  check  up  on  the  results  of  taken  test 
and  the  status  of  un-taken  tests  by  clicking  on  the  View  Test  Results 
button  of  the  main  OPS  TEST  Administration  form  (Figure  1.1).  When 
this  is  done,  they  get  a  query  form  which  can  be  used  to  select  what 
test  results  (or  pending  tests)  to  review.  The  selected  tests  are  then 
displayed  in  a  summary  form  from  which  they  may  be  printed  or  looked 
into  in  more  detail. 

Selecting  Test  Results  and  Status 

Clicking  on  the  View  Test  Results  button  of  the  main  OPS  TEST 
Administration  form  brings  up  the  Select  Specific  Test  Results  to  View 
form  shown  below  in  Figure  5.1. 


Figure  5.1:  Select  Specific  Test  Results  to  View  Form 
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On  the  Select  Specific  Test  Results  to  View  form  there  are  several 
fields  that  can  be  used  to  specify  selection  criteria  used  to  filter  the 
Specific  Test  results  to  be  reviewed.  All  such  specified  criteria  must  be 
satisfied  by  the  Specific  Test  results  returned.  These  selection  fields 
are: 

•  Test  Status:  The  test  status  can  be  one  (and  only  one)  of  the  three 
provided  options. 

•  The  “Any”  option  does  not  impose  a  selection  criteria  in  regards 
to  test  status. 

•  The  “Completed,  But  Not  Yet  Posted”  option  returns  those 
tests  that  have  been  completed  but  not  marked  as  having  their 
results  manually  posted  to  the  appropriate  personnel  records. 

•  The  “Not  Yet  Taken”  option  shows  all  the  specific  tests  that 
have  been  created  for  individual  crew  members  but  not  yet 
taken. 

•  Master  Test:  The  name  of  the  Master  Test  from  which  the  Specific 
Tests  to  be  reviewed  were  derived.  The  pull-down  list  provides  a 
listing  of  the  current  Master  Tests. 

•  Test-Taker:  Name  of  the  person  for  who  the  Specific  Test  was 
made. 

•  Administrator:  Name  of  the  standardization/evaluation/training 
Test  Administrator  who  created  the  Specific  Test. 

•  Position:  Crew  position  for  which  the  Specific  Test  was  designed. 

•  Compieted  Date  Range:  Range  of  dates  within  which  selected 
Specific  Tests  were  completed.  You  can  also  specify  only  a  first  (or 
a  last)  date  if  you  which. 

Once  the  selection  criteria  has  been  specified,  click  on  the  View 
Selected  Results  button  to  retrieve  those  Specific  Tests.  They  will  be 
displayed  in  summary  on  the  View  Specific  Test  Results  form. 

View  Specific  Test  Results  Form 

The  View  Specific  Test  Results  form  is  used  to  display  a  summary  of 
Specific  Tests  retrieved  via  the  just  discussed  query  form  and  is  shown  in 
Figure  5.2. 
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Figure  5.2:  Test  Results  and  Status  Summary  Form 

Each  returned  Specific  Test  results  displayed  on  the  View  Specific  Test 
Results  form  will  have  the  Master  Test  ID,  crew  member  tested,  crew 
position  they  tested  for,  date  the  test  was  completed  (if  it  is  completed), 
suspense  date  by  which  it  must  be  taken  (if  one  has  been  specified),  the 
score  (as  a  percentage  correct),  and  an  indication  as  to  whether  or  not  the 
results  have  been  posted  to  the  appropriate  personnel  records. 

For  each  Specific  Test,  there  are  also  buttons  to  get  a  print  of  that  test’s 
results  and  a  button  to  view  the  details.  The  Print  Results  button  causes  a 
summary  of  the  results  for  that  test  to  be  printed  on  the  default  printer.  The 
View  Details  button  brings  up  the  Specific  Test  form  for  that  test  and 
allows  you  a  range  of  further  functions  as  described  in  Chapter  4  of  this 
manual. 

The  Print  Summary  of  Results  button  at  the  bottom  of  the  form  produces 
a  summary  report  of  all  the  displayed  results  to  be  printed  on  the  default 
printer.  The  record  navigation  bar  at  the  bottom  of  the  form  allow  you  to 
move  through  the  Specific  Test  results  returned. 
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Chapter 


Test  Item  Anal^is 

How  do  you  analyze  the  results  of  using  test  items? 


Individual  test  items  in  the  OPS  TEST  testbank  can  be  analyzed  per 
their  past  performance  to  assess  their  quality  and  to  determine  if  they 
should  be  included  in  new  tests.  This  can  be  done  in  two  primary  ways. 
First,  when  an  individual  test  item  is  being  displayed,  there  is  a  button 
with  a  chart  on  it  that  leads  to  the  analysis  statistics  for  that  item.  The 
other  way  is  via  the  Test  Item  Analysis  button  on  the  OPS  TEST 
Administration  main  menu  (Figure  1.1),  which  leads  to  a  query  form. 

You  can  use  this  query  form  to  select  a  set  of  test  items  and  then  view 
the  analysis  statistics  for  each  of  these  items  in  turn.  This  chapter 
discusses  the  query  form  and  the  analysis  statistics  form. 

Selection  of  Test  Items  to  Analyze 

Clicking  on  the  Test  Item  Analysis  button  of  the  OPS  TEST 
Administration  main  menu  brings  up  the  Select  Test  Items  for  Analysis 
form  shown  in  Figure  6.1  below.  You  can  specify  whatever  combination 
of  selection  criteria  that  you  wish  and  the  returned  test  items  must  meet 
all  the  specified  criteria  (with  the  exception  that  if  both  Center  and 
Position  are  specified  and  not  in  the  same  center,  items  satisfying  either 
criteria  will  be  returned).  Several  of  the  selection  fields  have  pull-down 
lists  of  acceptable  values.  Some  allow  the  use  of  wildcards  (“?”  and  “*”) 
to  substitute  for  one  or  many  characters.  Other  fields  allow  a  range  of 
values  to  be  specified.  In  any  of  the  range  criteria,  either  the  lower, 
upper,  or  both  ends  of  the  range  can  be  specified.  There  are  range 
criteria  for: 

•  Item  Numbers:  The  unique  Testbank  Item  Numbers  assigned  to 
each  test  item. 

•  Test  Results:  Date  range  on  when  the  item  was  taken  in  a  test. 

•  Number  of  Misses:  Range  of  the  number  of  times  the  test  item 
was  incorrectly  answered. 
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•  Overall  Ease:  Range  on  the  overall  percentage  of  times  when  the 
test  item  was  correctly  answered. 

To  get  an  explanation  of  any  selection  criteria  field,  whether  wildcards 
are  allowed,  and  any  required  formatting  constraints,  simply  place  your 
cursor  over  the  field  and  a  caption  box  will  appear  with  this  additional 
information. 


Figure  6.1 :  Selecting  Test  Items  for  Analysis 

Test  Item  Analysis  Form 

Whether  you  ask  for  analysis  statistics  on  just  a  single  test  item  or  if  you 
select  a  set  of  items  to  analyze,  the  analysis  statistics  will  be  shown  for  a 
single  test  item  at  a  time  via  the  Test  Item  Analysis  form  shown  in  Figure 
6.2  below. 

There  are  three  parts  to  this  form.  Across  the  top  of  the  form  are  the 
unique  Testbank  Item  #  assigned  to  the  test  item,  the  correct  answer,  an 
overall  ease  (percentage  of  times  the  item  was  correctly  answered),  and 
counts  of  the  total  number  of  times  it  was  used  and  how  many  respondents 
chose  each  option.  The  correct  option  is  also  boxed  for  your  reference. 

The  second  part  of  the  form  provides  results  by  crew  position  and  includes 
the  position  as  well  as  ease  and  counts  for  when  the  test  item  was  taken 
for  that  crew  position. 
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The  third  part  of  the  form  provides  a  historical  listing  of  each  time  that  test 
item  has  been  used  in  a  Specific  Test.  This  listing  includes  the  date  of 
usage,  test  it  was  taken  as  part  of,  crew  position,  individual  answering  the 
test  item,  and  the  option  they  selected. 

The  View  Test  Item  button  is  provided  at  the  bottom  of  the  form  to  allow 
access  to  the  actual  test  item.  There  is  also  a  record  navigation  bar  at  the 
bottom  of  the  form  for  use  in  moving  between  test  items  when  multiple 
items  were  selected  for  analysis. 


Figure  6.2:  Test  Item  Analysis  Form 
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Chapter 


Formal  Test  Taking 

How  do  crew  members  lake  formal  baining  and 
evaluation  tests  on-line? 

Formal  training  and  evaluation  tests  can  be  taken  on-line  using  the  Specific 
Tests  created  by  Test  Administrators  per  the  instructions  in  Chapter  4. 
These  tests  are  specific  to  the  crew  member  taking  the  test  and  have  a 
password  protection  mechanism.  Once  the  test  administrator  has  created 
the  test,  they  will  give  the  test-taker  the  password.  The  test-taker  will 
access  the  test  via  the  instructions  below  and  work  through  the  test  items 
in  the  test.  When  done  with  the  test,  the  test-taker  clicks  on  the  a  Grade 
Test  button  at  which  point  the  test  is  graded  and  the  test-taker  is  given  the 
opportunity  to  review  and  get  feedback  on  any  test  item  that  they  may  have 
missed.  If  a  test  is  interrupted  once  it  has  been  started,  the  test-taker  may 
not  be  able  to  get  back  into  the  test  without  the  intervention  of  the  Test 
Administrator  to  “reset”  the  test  per  the  instructions  in  Chapter  4.  This 
depends  on  whether  the  “Can  Re-Enter?”  box  was  checked  for  the  test. 
Regardless,  if  the  test  has  been  graded,  the  Test-Taker  cannot  re-enter  it 
without  the  test  administrator’s  assistance. 

Starling  a  Formal  Test 

Once  a  crew  member  to  be  tested  has  been  given  the  password  to  their 
test  they  can  access  the  main  Test-Taking  menu  shown  in  Figure  7.1 . 
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Figure  7.1 :  Open  a  Specific  Formal  Test 

This  can  be  accessed  by  the  START|PROGRAMS|OPS  TEST  TEST¬ 
TAKING  menu  from  Windows. 

The  test-taker  then  clicks  on  the  Take  a  Formal  Training/Evaluation  Test 
button.  This  will  cause  a  Begin  Test  form  to  come  up  as  shown  in  Figure 
7.2.  You  can  then  locate  the  “Test”  in  the  pull-down  menu,  enter  the 
provided  password,  and  click  the  Begin  Test  button. 


Figure  7.2:  Selecting  a  Specific  Formal  Test  to  Open 
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Once  the  test  has  been  successfully  entered  you  will  get  the  individual  test 
items  displayed  one  at  a  time  via  the  Crew  Member  Test-Taking  form 
shown  in  Figure  7.3. 

Answering  Formal  Test  Items 

To  answer  a  test  item  in  a  formal  test,  click  on  the  appropriate  radio  button 
in  the  Answer  Here  selection  box  in  the  lower  left  corner  of  the  form.  You 
can  move  to  the  next  test  item  using  either  of  the  right  arrow  buttons 
provided  on  the  form.  The  first  right  arrow  button  (with  the  shadowed 
effect)  takes  you  to  the  next  unanswered  test  item.  The  plain  right  arrow 
takes  you  to  the  next  test  item  regardless  of  whether  or  not  it  has  been 
already  answered.  The  left  pointing  arrow  moves  you  to  previous  test 
items  and  the  one  with  the  shadowed  effect  moves  you  to  the  next 
previous  test  item  that  has  not  yet  been  answered.  If  the  test  item  has  an 
associated  multimedia  clip,  the  Play  Multimedia  button  will  be  available. 
Click  on  this  button  to  have  the  multimedia  clip  played  for  your  viewing. 
You  may  play  the  clip  as  many  times  as  you  like. 


Figure  7.3:  Formal  Test  Taking  Form 

Once  you  have  completed  the  test,  click  on  the  Grade  Test  button.  If  you 
have  left  any  test  items  unanswered  it  will  warn  you  and  provide  you  a 
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chance  to  go  back  and  answer  them  prior  to  grading.  Otherwise  your  test 
will  be  scored  and  your  score  presented  to  you.  If  you  missed  any  test 
items,  you  will  be  given  the  opportunity  to  go  back  and  review  them. 

When  reviewing  missed  test  items,  you  will  not  be  able  to  change  your 
answers  but  you  can  use  the  arrows  to  move  through  the  missed  items. 
With  each  missed  test  item  you  will  be  provided  the  governing  reference  for 
that  item  and  perhaps  some  additional  feedback  information  about  the  test 
item. 

Completing  Formal  Tests 

After  the  test  has  been  graded,  per  the  instructions  above,  and  any  missed 
items  optionally  reviewed,  simply  close  out  of  the  remaining  forms  to  exit 
OPS  TEST  Test-Taking. 
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Chapt^ 


Monthly  training  Tests 

How  do  you  lake  tie  Montily  training  Tests? 


Crew  members  can  use  OPS  TEST  Test-Taking  to  take  monthly  training 
tests  that  are  built  for  them  by  their  training  personnel.  These  tests  are  built 
upon  request  and  some  can  be  closed  without  being  graded  and  then  later 
re-entered  with  all  the  previous  responses  kept  intact.  The  status  and 
results  of  these  tests  are  typically  monitored  by  and  reported  to  the  training 
personnel. 

Taking  a  Monthly  Training  Test 

When  a  crew  member  wants  to  take  a  monthly  training  test,  they  can  do  so 
by  clicking  on  the  Take  Monthly  training  Test  button  of  the  main  Test- 
Taking  form  (seen  as  Figure  7.1).  This  can  be  accessed  via  the 
START|PROGRAMS|OPS  TEST  TEST-TAKING  menu  from  Windows. 

Clicking  on  the  Take  Monthly  Training  Test  button  on  this  main  Test- 
Taking  form  brings  up  a  form  that  allows  the  crew  member  to  specify  the 
test  they  wish  to  take,  their  name,  and  the  password  for  the  test  (if  one  has 
been  placed  on  the  test).  This  same  form  is  used  to  re-enter  tests  that 
were  started  but  not  completed.  This  form  is  shown  in  Figure  8.1  below. 
Once  this  information  has  been  specified,  click  on  the  associated  Take 
Test  or  Re-Enter  Test  button  to  access  the  test. 
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Figure  8.1 :  Selecting  MQF  Test  Items 

Answering  Monthly  Training  Test  Questions 

Once  you  have  gained  access  to  the  training  test,  the  test  items  are 
displayed  one  at  a  time  using  the  form  shown  in  Figure  8.2  below.  If  there 
is  a  multimedia  clip  associated  with  the  test  item,  the  Play  Multimedia 
button  will  be  shown  and  you  can  click  on  it  to  play  the  clip  as  many  times 
as  you  like.  Answer  the  test  items  by  selecting  the  appropriate  response  in 
the  Answer  Here  selection  box  in  the  lower  left  corner  of  the  form. 

As  with  formal  testing,  you  can  move  freely  between  test  items  using  the 
arrow  buttons.  The  ones  with  the  shadows  move  to  the  previous/next  un¬ 
answered  item  while  the  plain  arrows  move  you  to  the  immediate  prior/next 
item. 

Once  you  have  answered  all  the  questions,  you  can  click  on  the  Grade 
Test  button  to  have  your  test  scored.  If  you  answered  any  questions 
incorrectly,  you  will  be  provided  the  opportunity  to  review  them  with  the 
correct  answer  and  some  feedback  text  shown  for  you.  Some  test  items 
also  include  hyperlinks  to  associated  reference  material. 

If  you  wish  to  exit  the  test  prior  to  completing  it,  you  can  click  on  the  Close 
Form  button.  Depending  on  how  the  test  was  created,  you  may  or  may 
not  be  able  to  re-enter  it  later.  If  this  option  is  not  available  on  your  test, 
you’ll  need  to  have  your  training  test  administrator  reset  the  test  for  you. 
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Figure  8.2:  Answering  Test  Items 
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Chapter 


Oi>4jne  Access  to  Master 
Questioii  File 

How  do  you  access  the  Master  Question  File  to  take 
practice  tests  on-line? 


Crew  members  can  use  OPS  TEST  to  call  up  practice  test  items  from  the 
Master  Question  File  (MQF).  Practice  test  items  are  retrieved  via  a  query 
form.  The  crew  member  may  answer  as  many  or  as  few  of  the  retrieved 
test  questions  as  they  wish.  None  of  their  responses  will  be  recorded  or 
can  be  traced  back  to  them.  As  each  retrieved  MQF  test  item  is  answered, 
the  test-taker  is  given  immediate  feedback  as  to  whether  they  answered 
correctly  or  not.  If  they  did  not  correctly  answer,  the  test  item  is  displaced 
with  some  additional  reference  information  (including  the  correct  answer) 
so  they  can  learn  the  objective  of  the  question.  Upon  completing  a  set  of 
retrieved  questions,  an  overall  score  for  that  set  of  test  items  is  provided. 
At  this  point  the  crew  member  can  either  retrieve  some  more  MQF  test 
items  or  exit. 

Selecting  MQF  Questions 

When  a  crew  member  wishes  to  answer  some  practice  test  items  from  the 
MQF,  they  can  access  the  main  Test-Taking  form  (seen  as  Figure  7.1). 
This  form  is  accessed  via  the  START|PRQGRAMS|QPS  TEST  TEST¬ 
TAKING  menu  from  Windows. 

Clicking  on  the  Practice  Test  from  the  MQF  button  on  this  main  Test- 
Taking  form  brings  up  a  query  form  that  can  be  used  to  select  questions 
from  the  MQF  (see  Figure  9.1  below).  Several  of  the  selection  fields  allow 
the  use  of  wildcards  (“?”  and  “*”)  to  substitute  for  one  or  many  of  any 
characters.  You  can  aiso  find  current  values  for  these  fields  using  the  pull¬ 
down  lists  provided.  The  test  items  returned  must  meet  all  specified 
criteria.  Clicking  on  the  Take  Practice  Test  button  will  retrieve  the 
selected  test  items  and  display  them  one  at  a  time  to  be  answered. 
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Figure  9.1 :  Selecting  MQF  Test  Items 

Answering  MQF  Questions 

Once  test  items  have  been  retrieved  from  the  MQF  via  the  query  form 
described  above,  they  are  displayed  one  at  a  time  using  the  MQF  Test 
Taking  form  shown  in  Figure  9.2  below.  If  a  multimedia  clip  is  associated 
with  the  test  item,  the  Play  Multimedia  button  will  be  shown  and  you  can 
click  on  it  as  many  times  as  you  want  to  play  the  clip.  Answer  the  test 
items  by  selecting  the  appropriate  response  in  the  Answer  Here  selection 
box  in  the  lower  left  corner  of  the  form.  Your  response  will  be  immediately 
checked  against  the  correct  answer  and  you  will  be  told  if  you  got  it  right.  If 
you  missed  it,  you  will  be  provided  some  feedback  information  on  the 
question  and  what  the  correct  answer  is.  You  will  also  be  provided  with  a 
hyperlink  to  an  associated  reference  for  some  test  items.  Clicking  on  the 
link  will  open  that  reference  up  for  you. 

Qnce  you  have  answered  all  the  questions,  you  will  be  given  a  summary 
score  for  that  set.  You  can  then  select  more  questions  or  simply  exit  out  of 
the  form(s)  via  the  open  door  buttons  in  the  lower  right  corner.  None  of 
your  responses  to  MQF  questions  will  be  recorded  or  are  trace-able  back 
to  you. 
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MQF  Test-Taking 

■js*  -  ui  jii » '  ^  — 

1(U/RC) 

A  Mer 


The  multimedia  clip  shows  a  liftoff  of  what  NAS  A  Manned  Space  Fli^tProg'am  vehicle? 
(Click  the  PLAY  MULTIMEDIA  button  below) 


A.  Mercury 

B.  Gemini 


C.  ApoUo 


D  Space  Transportation  System 


Z] 


Your 

Answor 

^  A  B  C  D 

Play  Multimedia 

Figure  9.2:  Answering  MQF  Test  Items 
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Chapter 


Establishing  and 
Maintaining  Configuration 
of  the  OPS  TEST 


How  do  you  initially  establish  and  later  maintain  the 
OPS  TEST  database? 


When  you  initially  install  the  Operational  Test  and  Evaluation  Support  Tool 
(OPS  TEST)  on  your  network/computer(s),  you  need  the  backend 
database  installed  and  the  initial  configuration  of  several  items  established 
before  administrators  and  test-takers  can  begin  using  OPS  TEST. 

Installing  the  Back-End  Database 

Installation  of  the  OPS  TEST  backend  database  involves  copying  an  empty 
version  of  the  database  into  a  specified  location  and  file  name  on  a  shared 
network  drive.  Once  that  has  been  done,  you  should  install  one  copy  of  the 
OPS  TEST  Administration  program.  Upon  start  up,  it  will  detect  that  it  does 
not  have  the  current  path/location  for  the  OPS  TEST  database  and  prompt 
you  to  specify  the  new  location.  From  the  OPS  TEST  Administration 
program,  you  should  use  the  Configuration|Backend  Database  Password 
menu  selection  to  specify  a  new  password  for  the  backend  database  (see 
details  at  end  of  this  chapter).  You  will  then  give  this  password  together 
with  the  path  and  file  name  for  the  backend  database  to  everyone  that  is 
installing  either  or  both  the  OPS  TEST  Administration  and  Test-Taking 
software  as  they  will  be  prompted  for  this  information  when  they  first  start 
up  their  OPS  TEST  programs.  The  IITA  Technical  Support  personnel  can 
easily  assist  you  with  this  process. 
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OPS  TEST  Configuration  Setting 

Prior  to  usage,  OPS  TEST  requires  information  on  each  administrator  and 
test-taker,  codes  and  descriptions  for  tasks  that  will  be  tested,  designation 
of  operational  centers  (you  may  have  only  one),  designations  of  crew 
positions,  settings  for  the  menus,  ,  password  for  the  backend  database, 
and  password  for  accessing  menu  settings.  Furthermore,  you  may  want  to 
later  update  these  settings.  All  of  these  functions  are  provided  in  the 
Configuration  menu  from  within  the  OPS  TEST  Administration  software. 
Below  each  of  these  configuration  tasks  are  further  detailed. 

Update  Personnel  Info 

Probably  the  most  used  configuration  task  will  be  to  update  personnel 
information  in  the  OPS  TEST  database  to  include  adding  new  personnel. 
To  accomplish  this,  click  on  the  Configuration! Update  Personnel  Info  option 
from  the  menu  bar  in  the  OPS  TEST  Administration  software  (either  the 
standardization/evaluation  or  training  versions).  This  will  bring  up  the 
Configure  Personnel  Info  form  shown  in  Figure  10.1.  To  find  a  particular 
person’s  data,  type  in  or  search  for  their  name  in  the  Find  Person  list  box 
and  click  the  Find  Person  button.  You  can  now  modify  that  person’s  data 
as  needed.  In  addition  to  basic  name,  rank,  and  service  information,  there 
is  a  field  to  specify  which  center  they  normally  work  in  (although  one  is 
specified  here,  this  does  not  restrict  the  centers  they  can  test  for).  There 
are  also  check  boxes  to  specify  if  this  person  is  an  evaluator  and/or  a 
trainer,  which  allows  them  to  show  up  as  the  administrator  of  tests. 
Buttons  are  provided  on  this  form  to  allow  you  to  navigate  through  the 
personnel  info  records,  add  new  records,  delete  records  (although  we 
typically  don’t  delete  departed  personnel  so  we  can  keep  their  performance 
data  on  particular  test  items),  and  to  save  and  exit  the  form.  Hovering  the 
cursor  over  any  of  these  buttons  will  cause  a  caption  box  to  appear  which 
will  specify  the  button’s  function.  Moving  to  other  records  or  closing  the 
form  will  cause  any  changes  or  additions  to  be  automatically  saved. 
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Figure  10.1:  Configure  Personnel  Info  form 

Configure  Task  Codes 

Another  common  configuration  procedure  is  to  update  the  list  of  tasks  that 
are  being  tested.  These  tasks  include  a  task  code  (of  up  to  12  characters) 
and  a  description  of  the  task.  The  Configure  Task  Codes  form  (Figure 
10.2)  has  fields  for  both  the  task  code  and  the  description.  There  is  also  a 
Go  to  Task  selection  box  where  you  can  type  in  or  search  for  the  task  of 
interest  and  then  click  the  Go  to  Task  button  to  move  to  that  task.  The 
software  automatically  saves  updates  to  existing  tasks  and  the  addition  of 
new  tasks  when  you  move  to  a  new  task  or  close  the  form.  Other  buttons 
are  provided  for  you  to  move  between  tasks,  add  a  new  task,  delete  an 
existing  task,  explicitly  save  your  updates,  and  to  exit  the  form. 


Figure  10.2:  Configure  Task  Codes  form 
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Configure  Centers 

OPS  TEST  is  designed  to  accommodate  facilities  that  have  multiple 
operations  centers  (which  could  also  be  thought  of  as  groups  of  positions). 
The  Configure  Centers  form  (Figure  10.3)  allows  you  to  enter  and  modify 
the  names  for  these  centers.  As  with  the  other  forms  there  are  the  typical 
suite  of  buttons  to  allow  you  to  navigate  between  different  centers  and  to 
add/delete  centers  as  well  as  to  explicitly  save  updates  and  close  the  form. 
If  you  do  not  have  distinct  centers,  you  should  just  define  one  center, 
probably  as  your  organization’s  name,  and  use  that  one  center  for  all 
positions. 


Figure  10.3:  Configure  Centers  form 

(Configure  Crew  Positions 

Most  implementations  of  OPS  TEST  will  need  to  support  multiple  positions 
or  task  areas.  The  Configure  Crew  Positions  form  (Figure  10.4)  allows 
you  to  specify  and  update  these  positions.  With  each  position,  you  specify 
the  associated  center  and  then  the  position  name  or  descriptor.  The  center 
must  be  one  that  has  already  been  specified  using  the  Configure  Centers 
form.  If  you  have  not  already  specified  the  center,  you  can  use  to  Add 
Center  button  to  access  the  Configure  Centers  form.  As  with  the  other 
configuration  forms,  there  is  the  standard  suite  of  record  navigation  and 
maintenance  buttons. 


Figure  10.4:  Configure  Crew  Positions  form 
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Configure  Menu  Settings 

The  Configure  Menu  Settings  form  (Figure  10.5)  provides  a  mechanism  for 
you  to  configure  the  appearance  of  the  OPS  TEST  forms  so  as  to 
customize  your  implementation.  When  accessing  this  form,  you’ll  need  a 
password  that  has  been  hard-coded  into  your  implementation  of  OPS 
TEST.  This  restricts  access  to  this  global  function.  On  the  Configure  Menu 
Settings  form,  you  can  specify  the  security  classification  to  be  displayed  on 
test  item  forms  which  should  be  the  highest  classification  of  any  test  item. 
You  can  also  specify  what  you  call  your  task  codes.  In  the  example  of 
Figure  10.5,  we  just  call  them  “Task”,  but  other  possible  names  are  “PEC” 
for  Performance  Evaluation  Criteria  or  “JPR”  for  Job  Performance 
Requirement.  There  are  also  fields  to  specify  a  form  caption  (which  is 
prefixed  to  several  of  the  forms’  captions)  and  a  foreword  that  describes 
the  program.  When  these  have  been  specified/updated,  you  can  either 
explicitly  save  them  via  the  Save  button  or  just  close  the  form,  which  will 
also  cause  the  updates  to  be  saved.  It  is  possible  to  specify  multiple 
configuration  settings,  but  only  the  first  one  will  actually  be  used  by  OPS 
TEST. 


Figure  10.5:  Configure  Menu  Settings  form 

Configuration  Password 

Selection  Configuration|Configuration  Password  allows  you  to  change  the 
password  that  protects  the  Configure  Menu  Settings  form  (Figure  10.5).  To 
accomplish  this  you  will  need  to  specify  the  current  (old)  password  and 
then  the  new  password  twice.  Feedback  is  provided  regarding  whether  the 
password  was  successfully  changed  or  not.  The  maximum  password 
length  is  12  characters  and  the  default  password  that  the  OPS  TEST 
Administration  software  comes  with  is  “config4amos”. 
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Backend  Database  Password 

The  Configuration  I  Backend  Database  Password  selection  provides  access 
to  a  form  that  allows  you  to  change  the  password  on  the  OPS  TEST 
backend  database  where  all  the  test  items,  tests,  and  results  are  stored. 
This  is  a  single  common,  shared  database  used  by  all  OPS  TEST 
programs  that  typically  resides  on  a  network  disk  drive.  When  you  select 
this  function,  the  form  shows  the  current  password.  You  then  type  in  your 
new  password  (which  must  be  exactly  six  digits  (0-9))  and  click  the  Update 
Password  button.  You  will  be  told  if  the  operation  was  successful  or  not. 
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